Parallelization and Load Balancing Techniques for HPC

As multicore systems become ubiquitous in desktop, and in mobile and embedded systems, interest in high performance computing (HPC) techniques has increased. Further, several computation intensive tasks demand use of high performance computing resources (Raju et al., 2009, Pande et al., 2009, Varré et al., 2011, Gupta et al., 2008) since sequential computing platforms are proving to be incapable of fulfilling the computational demands in these domains. Hence, researchers are using parallelization techniques. However, parallelization also brings the need of achieving load-balancing, since an unbalanced load distribution is likely to lead to wastage of processors and increased total completion time. In this chapter, we discuss three different scheduling techniques which are used for achieving load-balancing. These techniques are static scheduling, master-slave scheduling and work-stealing. To show a concrete example of parallelization approach, we show the example of multi-threading in Java (Arnold et al., 2000). We discuss the relative advantages and disadvantages of multi-threading implementation in Java.